Update: I apologize for my confusion about the dynamics. I feel more positively now about this work, and have increased my score. There are two issues here to be addressed: a) how realistic is it for the dynamics of the feedforward pass recurrence within layers to run to convergence *before* sending down the top-down feedback? What happens if these are concurrent processes, such that units get both bottom-up and top-down inputs at the same time? Given the time-scale of the recurrent dynamics in cortex, the authors could then ask (in their model) whether this delay is "enough" for their push-pull mechanism to work. If yes, that would strengthen the result a fair bit.

In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a two-layer Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on the CoNLL-2000 dataset reveal a notable improvement over existing unsupervised methods, enhancing phrase F1 score by up to 6 percentage points. Further, finetuning with downstream tasks results in an additional performance improvement. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model's downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.